Computer  Science 

AD-A285  210  ! 


Integrating  Human  Factors  with 
Software  Engineering  Practices 

William  E.  Hefley,  Elizabeth  A.  Buie,  Gene  F.  Lynch, 
Michael  J.  Muller,  Douglas  G.  Hoecker, 

Jim  Carter,  J.  Thomas  Roth 

29  July  1994 
CMU-CS-94-175 


m-* 


i  v, L- 

|;w  '-V 


»‘Ppfjp  **?■ 

Lift  W 


V\ 


ellon 


'.i.  .-1  i'1  ' 

./  •-  i  \  v',C'  ,  (lift  ’  t  .> 
.i!  i  V 


IjDTIC 

4WELECTE  | 
^PCTiO.S:  19341 


4PP iwtf  lor  pnbtta  ralw 
DWribottoB  UnJbattM 


B  4f  O' 


04.  Q 


Integrating  Human  Factors  with 
Software  Engineering  Practices 


William  E.  Hefley,  Elizabeth  A.  Buie,  Gene  F.  Lynch, 
Michael  J.  Muller,  Douglas  G.  Hoecker, 

Jim  Carter,  J.  Thomas  Roth 

29  July  1994 
CMU-CS-94-175 


School  of  Computer  Science 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

To  appear  in  the  Proceedings  of  the  38th  Annual  Meeting  of  the 
Human  Factors  and  Ergonomics  Society,  Nashville,  TN,  October,  1994. 

Also  appears  as  Human-Computer  Interaction  Institute  Technical  Report 
CMU-HCn-94-103 


Abstract 

Engineering  processes  and  methodologies  used  in  building  tomorrow's  systems  must  place  a  greater  emphasis 
on  designing  usable  systems  that  meet  the  needs  of  the  systems’  users  and  their  tasks.  This  paper  identifies 
the  need  for  defining  human  factors  and  human-computer  interaction  (HCI)  engineering  activities  that 
contribute  to  the  design,  development,  and  evaluation  of  usable  and  useful  interactive  systems,  and  presents  a 
rationale  for  integrating  these  activities  with  software  engineering  and  incorporating  them  into  the  system  life 
cycle. 
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1.  Introduction 

The  design  and  development  of  human-computer  interaction  (HCI)  has  been  evolving 
into  a  full  engineering  discipline  for  achieving  system  usability —  developing  systems  that 
support  their  users  in  accomplishing  their  tasks  with  effectiveness,  efficiency,  and 
satisfaction.  Advances  have  been  occurring  both  in  user  interface  engineering  (Curtis  & 
Hefley,  1994),  focusing  on  the  processes  being  used  to  develop  artifacts,  and  in  usability 
engineering  (Whiteside,  Bennett  &  Holtzblatt,  1988;  Nielsen,  1993),  focusing  on  the 
products  being  developed. 

Large  proportions  of  these  systems  are  heavily  software  intensive.  System  engineering 
activities  must  therefore  integrate  HCI  engineering  with  software  engineering  to  achieve 
usability  in  software-intensive  systems.  This  effort  can  take  advantage  of  successful  HCI 
engineering  efforts,  which  have  focused  on  human  factors  and  HCI  methods  (Dayton  et  al., 
1993). 

Just  as  software  engineering  is  becoming  a  discipline  with  a  defined,  managed  process, 
HCI  engineering  is  continuing  to  evolve  to  a  discipline  having  its  own  defined  interface 
development  processes.  These  processes  will  be  practiced  by  people  from  numerous  fields 
employing  a  rich  collection  of  analytical,  design,  development,  and  evaluation  techniques 
to  develop  interactive  systems  that  are  effective,  efficient,  and  satisfying.  Successful  HCI 
engineering  efforts  often  focus  on  human  factors  and  human-computer  interaction  methods 
that  can  improve  the  practice  of  software  engineering. 

How  can  HCI  enhance  current  product  development  practices?  As  a  step  in  integrating 
HCI  engineering  with  software  engineering,  this  paper  addresses  questions  of  how  HCI 
engineering  principles  and  practice  can  enhance  current  software  engineering  practice. 

This  paper  describes  the  background  and  motivation  for  a  seminar  to  be  held  as  part  of 
the  HFES94  Annual  Meeting.  This  working  session  has  two  main  objectives: 


•  Identify  engineering  methods  and  techniques  appropriate  to  the  HCI  engineering 
process  model  for  interactive  systems  development 

•  Define  the  processes  for  using  these  techniques  in  the  context  of  a  system  life  cycle-a 
process  architecture  that  elaborates  on  how  to  specify,  design,  build,  test,  and  evaluate 
useful,  usable,  and  satisfying  interactive  systems. 

These  efforts  aim  to  propose  HCI  engineering  processes  for  interactive,  software¬ 
intensive  systems  and  to  extend  their  understanding  of  these  techniques,  methods,  and 
processes  to  a  broader  community  of  researchers  and  practitioners.  We  hope  that  these 
ideas  may  mature,  spread,  and  begin  to  change  the  software  and  system  engineering 
practices. 

2.  State  of  the  world 

2.1  People 

Who  performs  these  types  of  tasks  today?  What  skills  do  they  have  or  need  (Dayton, 
1993)?  Are  they  being  brought  in  early  enough  as  an  integral  part  of  the  development 
process  (Whitehurst,  1993;  Rousseau,  Candy  &  Edmonds,  1993)?  Developing  interactive 
systems  requires  the  timely  acquisition  and  application  of  new  skills  to  comprehend,  apply 
and  improve  concepts  in  development  processes,  methods,  and  tools. 

2.2  Processes 

An  important  deficiency  in  the  current  state  of  practice  is  that  many  HCI  design 
methods  are  poorly  defined.  A  common  criticism  of  software  engineering  is  also  that  its 
processes  have  not  yet  reached  the  level  of  discipline  and  proceduralization  that  are  evident 
in  other  engineering  disciplines.  The  processes  used  in  developing  large,  complex  systems 
are  often  ad  hoc,  and  not  often  defined  and  articulated  in  a  manner  that  encourages 
repetitive  use  and  further  refinement. 

However,  many  of  today's  state-of-the-practice  software  engineering  organizations  are 
assessing  the  maturity  of  the  processes  they  use  and  are  putting  into  place  various  forms  of 
continuous  process  improvement  (also  called  Total  Quality  Management  [TQM])  activities 
to  plan  and  carry  out  improvements  to  their  existing  software  development  processes 
(Herbsleb  &  Zubrow,  1994).  Such  organizations  are  striving  to  make  their  software 
processes  understood,  repeatable,  defined,  measured  and  subject  to  continual  improvement. 
They  are  addressing  a  vital  need  for  process  architectures  that  are  usable  by  large  numbers 
of  practitioners  to  produce  high  quality  software  systems. 

Unfortunately  not  many  of  these  efforts  include  state  of  the  practice  in  HCI  design  in 
the  processes  being  improved.  Nor  do  they  use  the  evaluative  methods  for  determining 
usability  as  metrics  for  the  success  of  their  processes.  They  are  improving  process 
(software  and  to  a  lesser  extent  product  process),  but  it  is  a  critically  impaired  or  at  least 
structurally  limited  process  due  to  its  lack  of  consideration  for  HCI  design  and  usability 
issues.  Developing  high-quality  interactive  systems  requires  the  appropriate  integration  of 
HCI  engineering  with  software  engineering  during  the  entire  system  life  cycle. 
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3.  Designing  Tomorrow’s  Interactive  Systems 

3.1  Desired  Goal  State 

Not  only  must  future  systems  support  operability  and  learnability  goals,  they  must  also 
be  developed  with  an  eye  to  concerns,  such  as  affordability.  Do  our  engineering  processes 
result  in  a  system  that  people  and  organizations  can  afford  to  procure  and  operate?  Can  a 
usable  system  be  produced  within  the  schedule  and  cost  constraints  that  face  us  as 
developers? 

What  goal  should  HCI  engineering  and  software  engineering  adopt  in  this  context? 
They  must  aim  to  apply  a  coordinated  engineering  process  for  effectively,  efficiently, 
consistently,  and  humanely  producing  high-quality,  defect-free  products  that  fully  satisfy 
its  users’  needs. 

3.2  Software  Process  Improvement  as  a  Model 

Unfortunately,  because  of  recurring  problems  and  the  immaturity  of  many 
organizations,  the  major  process  emphasis  in  these  organizations  is  typically  on  planning, 
managing  and  controlling  the  progression  of  software  development  activities  (Humphrey, 
Kitson  &  Kasse,  1989;  Kitson  &  Masters,  1992).  Recently,  however,  the  software 
community  have  begun  to  pay  widespread  attention  to  ways  of  understanding  and 
improving  software  processes  (Humphrey,  1989;  Paulk,  1993a). 

The  Software  Engineering  Institute’s  (SEI)  Capability  Maturity  Model  (CMM)  (Paulk, 
1993a;  Paulk,  1993b)  defines  five  maturity  levels  for  software  process  and  describes  the 
processes  that  typically  are  in  place  in  organizations  at  each  process  maturity  level.  The 
CMM  provides  specific  guidance  on  the  staging  of  activities  in  software  process 
improvement.  This  structuring  of  specific  key  process  focus  areas  within  maturity  levels 
helps  organizations  prioritize  their  improvement  activities. 

The  notion  that  an  organization  is  improving  its  software  process  maturity  implies  that 
the  organization  is  moving  towards  defining,  applying  and  improving  rigorous 
development  processes.  Once  a  rigorous  development  process  architecture  is  defined,  its 
can  be  used  to  guide  development,  provide  a  common  basis  for  communication  among 
engineers  and  managers,  and  (perhaps  most  importantly)  provide  a  basis  for  further  process 
improvement  (Curtis  &  Hefley,  1992).  The  software  process  improvement  approach  can 
provide  a  valuable  model  for  organizations  seeking  to  improve  their  HCI  engineering 
process. 

33  Integrating  HCI  Engineering  Processes  with  Defined 
Software  Engineering  Processes 

Until  recently,  the  HCI  community  has  placed  primary  emphasis  on  design.  This 
emphasis  isn’t  bad-in  fact,  it's  to  be  expected  if  HCI  practice  is  at  an  ad  hoc  process 
maturity  level  (CMM  Level  1).  Curtis  and  Hefley  have  argued  ( 1992, 1994)  that  rigorous 
HCI  processes,  methods,  and  techniques  do  exist. 
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Why  try  to  define  an  engineering  process,  if  we're  really  only  at  a  Level  1  or  ad  hoc 
stage  of  building  interfaces?  As  discussed  in  greater  detail  below,  there  is  a  growing 
importance  of  HCI  concerns  in  today's  applications.  This  growing  importance,  coupled 
with  the  maturation  of  the  discipline,  present  a  timely  opportunity  to  make  improvements 
in  the  system  life  cycle  process  and  its  outcomes  by  focusing  on  human  factors  and  HCI 
concerns. 

Thus,  if  Level  3  is  characterized  by  defined  processes,  we  need  to  start  making  the  HCI 
engineering  processes  repeatable  (CMM  Level  2)  and  defined  (CMM  Level  3).  We  also 
need  to  consider  how  to  integrate  these  processes  into  a  coherent  engineering  process.  At 
the  same  time,  we  need  to  start  thinking  about,  and  developing  plans  for,  integrating  those 
HCI  engineering  processes  with  our  defined  software  engineering  processes. 

For  many  HCI  development  efforts,  necessary  HCI  engineering  processes  are  often  not 
integrated  with  software  engineering  processes  (Curtis  and  Hefley,  1992).  However,  HCI 
development  is  beginning  to  adopt  new  approaches  to  system  development  such  as  user 
centered  system  design  (Norman  &  Draper,  1986),  participatory  design  (Schuler  & 
Namioka,  1993;  Muller  &  Kuhn,  1993;  Greenbaum  &  Kyng,  1991),  participatory 
ergonomics  (Noro  &  Imada,  1991),  Scandinavian  design  (Bpdker,  1990;  Floyd,  1989),  and 
cognitive  modeling  (Olson  &  Olson,  1990;  Bosser  &  Melchior,  1992;  Tauber,  1990), 
which  make  the  end  user  rather  than  the  technology  the  focus  of  the  design  process.  These 
approaches,  together  with  an  increasing  awareness  of  the  need  for  process  architectures  that 
integrate  HCI  and  software  development  processes  (Curtis  &  Hefley,  1994;  Long,  et.  al, 
1994;  Lim,  Long  &  Silcock,  1990;  Browne,  1994;  Hix  &  Hartson,  1993),  signal  the 
emergence  of  process  architectures  that  will  be  useful  in  developing  usable  interactive 
systems. 

4.  Importance  of  HCI  Concerns 

The  development  of  computer-based  systems  is  changing.  The  changes,  many  of  which 
reflect  a  growing  recognition  of  the  importance  of  HCI  concerns,  include 

•  the  predominance  of  HCI-related  effort  in  the  life  cycle 

•  the  expanding  functionality  of  user  interfaces-moving  towards  intelligent  user 
interfaces  and  integrated  task  environments 

•  the  transition  of  HCI  development  from  an  arcane  specialty  into  an  established 
engineering  discipline. 

4.1  Predominance  of  HCI 

Almost  half  of  the  software  in  systems  being  developed  today  and  thirty-seven  to  fifty 
percent  (depending  on  life-cycle  phase)  of  the  efforts  throughout  the  life  cycle  (Myers  & 
Rosson,  1992)  are  related  to  the  system’s  user  interface.  A  significant  portion  of  the 
resources  and  efforts  in  software  development  are  dedicated  to  that  portion  of  the  systems 
commonly  referred  to  as  the  “user  interface.”  In  fact,  the  user  interface  has  become  a  core 
system  engineering  issue  separate  from  usability  concerns  (Curtis  &  Hefley,  1994). 
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Moreover,  the  key  constraints  on  HCI  development  are  evolving.  An  ever-widening 
population  of  potential  users  continues  to  make  ever-increasing  demands  for  usability. 
Formerly  constrained  almost  exclusively  by  technology,  HCI  development  is  now  driven 
mainly  by  usability  concerns  and  increasingly  by  concerns  regarding  operability  (including 
leamability). 

4.2  Expanding  HCI  Functionality 

The  functionality  and  capabilities  of  user  interfaces  are  also  expanding-moving,  for 
example,  toward  intelligent  user  interfaces  (IUIs)  (Hefley  &  Murray,  1994)  and  integrated 
task  environments  (ITEs)  (Hefley  &  Romo,  1994).  Concepts  such  as  critics  (Mastaglio, 
1990)  and  guides  (Tuck  &  Olsen,  1990),  rich  research  topics  a  few  short  years  ago,  are  now 
being  incorporated  in  shrink-wrapped  products. 

This  progression  can  be  expected  to  continue.  For  example,  addressing  integrated  task 
environments  (1TE)  the  United  States  Department  of  Defense  (DoD)  Software  Technology 
Strategy  (Department  of  Defense,  1991)  brings  out  the  concept  of  intelligent  adaptive  user 
interfaces  (IAUI).  This  strategy  addresses  needs  for  intelligent  adaptive  user  interfaces 
(IAUI),  safe  user  interfaces,  user-tailorable  interfaces,  task  model-based  interfaces, 
adaptive  user  interfaces,  and  intelligent  user  interfaces. 

In  all  categories  of  products  customers  demand  more  for  less.  They  want  more 
functionality  for  less  money.  They  want  more  flexibility  with  less  learning  or  execution 
effort. 

4 3  An  Engineering  Discipline  for  HCI 

During  the  1990s,  HCI  development  will  complete  the  transition  from  an  engineering 
specialty  into  an  engineering  discipline  (Curtis  fc  Hefley,  1994),  and  HCI  professionals  and 
“drafted”  engineers  will  find  themselves  becoming  a  more  important  part  of  the 
development  team.  They  will  also  find  that  this  requires  greater  discipline  in  their  work. 
This  discipline  is  not  just  technical,  it  also  involves  taking  greater  responsibility  for  serious 
analytical  activities  that  lead  to  a  finished  product. 

Curtis  and  Hefley  (1992,  1994)  have  argued  that  rigorous  processes,  methods  and 
techniques  do  exist  for  HCI  development  and  that  they  constitute  an  engineering  discipline. 
Others  (e.g.,  Morrison,  1993)  have  suggested  that  HCI  cannot  be  an  engineering  discipline; 
however,  we  believe  this  argument  to  be  based  on  an  overly  narrow  definition  of 
engineering.  In  recent  years,  other  colleagues  have  also  publicly  taken  a  stance  of  moving 
towards  an  engineering  discipline  (Dowell  &  Long,  1989;  Dumas  &  Redish,  1993). 

The  processes  used  by  system  designers  and  developers  are  aimed  at  producing  a  high- 
quality  product — a  product  that  enables  the  users  to  accomplish  their  tasks  efficiently, 
effectively,  and  comfortably.  In  a  disciplined  fashion,  designers  and  developers  must 
address  the  intended; 
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•  users  and  their  characteristics,  such  as  knowledge  and  skills 

•  users’  jobs  and  tasks,  including  task  objectives,  performance  needs,  and  interpersonal 
or  group  communication  needs 

•  organizational  &  work  environment 

•  quality  of  worklife  and  the  quality  of  users’  experience 

•  technologies  to  support  task  performance 

•  information  needed  by  users  and  their  tasks 

•  interrelationships  between  the  environment,  users,  tasks,  technologies,  and  information 
flows 

HCI  engineering,  as  a  discipline,  is  uniquely  able  to  contribute  to  addressing  these 
issues.  In  one  recent  analysis,  the  interaction  times  of  similar  functions,  based  on  detailed 
cognitive  task  analyses,  were  compared  for  two  different  proposed  workstations.  The  cost 
differential  for  a  0.8  second  performance  difference  in  each  transaction  spread  across  all 
operators  accounted  for  a  potential  savings  of  $2,400,000  per  year  (Gray,  John,  and 
Atwood,  1992).  Some  organizations  have  already  begun  to  define  HCI  engineering  as  an 
important  part  of  their  system  life  cycle  (Computer  Sciences  Corporation,  1990,  1994). 

5.  Envisioning  A  Future  Discipline 

While  HCI  engineering  is  continuing  to  evolve  into  a  discipline  having  its  own  defined 
interface  development  processes,  promising  advances  along  this  evolutionary  path  are  not 
only  found  in  an  evolving  discipline  for  designing  usable,  effective,  and  productive 
systems  with  concern  for  usability  at  multiple  levels — at  the  individual  ergonomic  level,  at 
the  task  and  job  level,  at  the  interpersonal  or  group  communication  level,  and  at  the 
organizational/societal  level — but  also  in  defining  ways  of  analyzing,  design,  and  assessing 
issues  directly  related  to  quality  of  worklife  (QWL)  (or  the  quality  of  life  for  non¬ 
workplace  applications)  and  the  users’  quality  of  experience.  Floyd  (1987)  foresaw  this 
paradigm  shift  in  understanding  software  in  its  contexts  of  human  learning,  work,  and 
communication. 

Key  to  successful  system  development  are  an  increasing  emphasis  on  defining 
appropriate  human  factors  and  HCI  design  processes  and  an  integration  of  these  processes 
with  existing  system  and  software  development  processes.  It  is  easy  to  see  the  value  of 
these  improvements  when  a  second  less  in  user  execution  time  yields  millions  of  dollars  in 
benefits  (Gray,  John,  and  Atwood,  1992);  however,  many  efforts  may  not  have  this  kind  of 
dramatic  outcome.  To  enjoy  the  resulting  benefits,  we  will  have  to  take  action  to  develop 
comprehensive  life  cycle  processes  that  are  repeatable,  defined,  cost  effective,  measurable, 
and  traceable.  A  process  that  can  be  evaluated  and  then  improved. 

This  paper  is  a  call  to  action  and  an  invitation  to  all  who  are  responsible  for  product 
development  and  see  the  need  for  integration  of  clearly  defined  processes  that  address  a 
range  of  analysis,  design  and  evaluation  functions  to  assist  in  developing  a  vision  of  future 
practice  that  will  encompass  designing  for  usability  (fit  to  person),  utility  (fit  to  the  task 
and  organization),  and  QWL  (fit  with  respect  to  social  considerations  regarding  people, 
groups,  organizations,  and  society). 
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